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REMARKS 

Claims 1-10 are pending in the present application. Claims 1, 3 and 6 are 
independent. Reconsideration of this application, in view of the following remarks, is 
respectfully requested. 

Rejection Under 35 U.S.C. § 103 

Claims 1 and 2 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Rankin et al, U.S. Patent No. 5,489,099 in view of Edelson et al., U.S. Application 
Publication No. 2002/0054211 A1. Claims 3 and 4-8 stand rejected under 35 U.S.C. § 
103(a) as being unpatentable over Rankin et al. in view of Yokota et al., U.S. Patent No. 
5,905,530. Claims 9 and 10 stand rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Rankin et al. in view of Edelson et al. and further in view of Yokota et al. 
These rejections are respectfully traversed. 

At the outset, as the Examiner will note, attached hereto is a verified translation of 
Japanese Application No. 2000-339896, which was filed on November 8, 2000. In view of 
this, the foreign priority of the present application has been perfected to November 8, 2000. 
Referring to the Edelson et al. reference relied on by the Examiner, this reference was filed 
on November 5, 2001 , which is after the perfected filing date of the present application of 
November 8, 2000. In view of this, the Edelson et al. reference is only available as a 
reference against the present invention with regard to the subject matter that is present in 
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U.S. Provisional Application No. 60/245,710 (hereinafter "the 710 Application), filed on 
November 6, 2000. Since the Examiner has not explained that the 710 Application 
includes the same subject matter as the Edelson et al. Published Application, Applicants 
respectfully submit that the Examiner has not established a prima facie case of 
obviousness. 

For the convenience of the Examiner, attached hereto is a copy of the 710 
Application. Applicants submit that the 71 0 Application does not include the same subject 
matter as the 710 Application and therefore, the Examiner's rejection is improper without 
an explanation with regard to where in the 710 Application there is sufficient teaching to 
modify the Rankin et al. reference to arrive at the presently claimed invention. Since the 
Examiner has not provided such an explanation, Applicants reiterate that the Examiner has 
not established a prima facie case of obviousness. 

It should also be noted that the 710 Application has a filing date of November 6, 
2000, which is only two days prior to the filing date of the present application of November 
8, 2000. If the Examiner does persist in the rejection in view of the Edelson et al. 
reference, Applicants reserve the right to file a Declaration to antedate the Provisional 
Application filing date of the Edelson et al. reference. 

In any event, Applicants present the following arguments against the Examiner's 
rejection, taking into consideration the disclosure in the 710 Application. With regard to 
independent claim 1 , this claim is directed to a ball motion measuring apparatus and recites 
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a combination of elements including "a calculating section for carrying out a magnifying 
process of only a portion of an original image including a ball image, thereby calculating 
magnified image data" and "a display section for displaying a magnified image based on 
the magnified image data wherein the magnified image data is used to calculate ball 
motion ." Applicants respectfully submit that the combination of Rankin et al. and Edelson 
et al. fails to render obvious the present invention as recited in independent claim 1 . 

In particular, referring to Rankin etal., this reference is directed to an apparatus and 
method for tracking the flight of a golf ball. Referring to column 3, lines 55-62 of Rankin et 
al., it is disclosed that an image controller 27 is used to focus the camera lens on the golf 
ball and control the zoom or magnification of the camera lens in order to maintain a 
consistent ball image size within a video frame. In view of this, there is no magnification 
process of an original image as recited in independent claim 1 of the present invention. In 
Rankin et al., there is only original image data. The focusing and zooming of the camera 
lens do not result in magnified image data, but result in original image data that is focused 
and closer to the ball than if the zoom feature was not included. As recognized by the 
Examiner, since the Rankin et al. reference only discloses original image data, the Rankin 
et al. reference fails to disclose the use of magnified image data to calculate ball motion as 
recited in independent claim 1 of the present invention. However, the Examiner relies on 
the Edelson et al. reference in order to modify Rankin et al. to arrive at the presently 
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claimed invention. Applicants respectfully submit that the Examiner's modification would 
not arrive at the presently claimed invention. 

In the Examiner's Office Action, the Examiner asserts that Edelson et al. discloses a 
"magnifying process for calculating motion" (see page 2, paragraph 3, lines 7-9 of the 
Examiner's Office Action). Although the Edelson et al reference ('710 Application) may 
disclose magnification of an image, Edelson etal. does not disclose calculating any motion 
with the magnified portion of the image. Referring to the 710 Application, this document 
describes a surveillance device. At page 3, paragraph 2 of this document, it is stated 
"[w]ith motion identified, moving objects can be highlighted, enlarged, tracked and 
alarmed." (emphasis added). In addition, on page 7, lines 6-8, it is stated "Binocular 
Effect - An area surrounding the object can be defined to be magnified. This area will 
have a zoom effect, magnifying the area by 2x or 4x while leaving the area outside the area 
unchanged." In view of this, the '710 Application appears to disclose "magnification" of 
image data. However, there is no description in the '71 0 Application of using the magnified 
image data to calculate any motion of the magnified image, and there is certainly no 
description in the '71 0 Application of calculating ball motion as recited in independent claim 
1 of the present invention. In view of this, Applicants submit that the Edelsen et al. 
reference is insufficient to modify Rankin et al. to arrive at the present invention as recited 
in claim 1. Specifically, any modification of Rankin et al. in view of Edelson et al. would 
only result in the image data being magnified. There is insufficient teaching in Edelson et 
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al. to modify Rankin et al. to magnify the image data and use the magnified image data to 
calculate ball motion. In view of this, the Examiner's rejection of claim 1 in view of the 
Edelson et al. reference is improper and should be withdrawn. 

With regard to dependent claims 2, 9 and 10, Applicants respectfully submit that 
these claims are allowable due to their dependence on independent claim 1 , as well as due 
to the additional recitations in these claims. 

With regard to independent claims 3 and 6, the Examiner recognizes that the Rankin 
et al. reference fails to disclose "a calculating section for correcting a coordinate error" as 
recited in independent claims 3 and 6 of the present invention. However, the Examiner 
relies on the Yokota et al. reference in order to modify Rankin et al. to arrive at the 
presently claimed invention. Applicants submit that the Yokota et al. reference fails to 
make up for the deficiencies of Rankin et al. 

It should be noted that independent claims 3 and 6 recite "a calculating section for 
correcting a coordinate error of only a ball image in the original image ." In addition, 
independent claims 3 and 6 recite " said correction data being used to calculate true 
coordinates of the ball image." As recognized by the Examiner, the Rankin et al. reference 
fails to disclose calculating any coordinate error and fails to disclose using correction data 
to calculate true coordinates of the ball image. In view of this, the only element of 
independent claims 3 and 6 that Rankin et al. discloses is a CCD camera and calculating 
motion of a ball from the ball image. Applicants respectfully submit that the Yokota et al. 
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reference certainly fails to make up for all of the deficiencies of Rankin et al. There is 
absolutely no suggestion in the Yokota et al. reference to calculate a coordinate error and 
use correction data to calculate ball motion as recited in independent claim 1 . 

Referring to Yokota et al., this reference discloses an image pickup apparatus that 
corrects distortion of an image due to an objective lens, not an object in an image. In 
addition, Yokota et al. fails to disclose calculating anything with regard to the corrected 
image. In view of this, Applicants submit that there is insufficient suggestion in Yokota et 
al. to modify Rankin et al. to both calculate a coordinate error to obtain correction data and 
use the correction data to calculate true coordinates of the ball image as recited in 
independent claims 3 and 6 of the present invention. 

The Yokota et al. reference is not directed to a camera, which is used for measuring 
any motion, in particular ball motion as recited in independent claims 3 and 6 of the present 
invention. In view of this, there is no suggestion in the Yokota et al. reference to use 
corrected data for calculating motion of an object. Therefore, there is insufficient 
suggestion to modify the Rankin et al. CCD camera to both calculate a coordinate error to 
obtain correction data of only a ball image and use the correction data to calculate true 
coordinates of the ball image as recited in independent claims 3 and 6 of the present 
invention. 

With specific regard to independent claim 6, the Examiner asserts that this claim is 
similar to claim 3 and therefore is also rejected in view of the Rankin et al. and Yokota et al. 
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references. Specifically, the Examiner asserts that the shift of a direction of the object 
image would be included in the process of correcting the object image distortion. However, 
independent claim 6 recites that the calculating section corrects a coordinate error of only a 
ball image in the original image "made by a shift of a direction of the ball image from a 
direction of an optical axis of the CCD camera." This aspect of the present invention has 
nothing to do with the distortion caused by the lens of the CCD camera. The coordinate 
error caused by "a shift of a direction of the ball image from the direction of the optical axis 
of the CCD camera" is due to the ball 1 being shifted from the reference plane (see the 
reference plane s in Figure 5 of the present invention). Referring to page 9, second full 
paragraph of the present invention, this aspect of the present invention is described. 
Specifically, if the ball 1 is shifted from the reference plane s due to a shot that is not 
straight, the z coordinate of the ball is not zero. Therefore, the coordinate error calculated in 
claim 6 is due to the ball being closer to or further away from the reference plane. In view 
of this, in addition to the above deficiencies of Yokota et al., Applicants submit that this 
reference also fails to disclose calculating a coordinate error due to a shift of a direction of 
the ball image from a direction of the optical axis of the CCD camera as recited in claim 6. 

With regard to dependent claims 4, 5, 7 an 8, Applicants respectfully submit that 
these claims are allowable due to their respective dependence on independent claims 3 
and 6, as well as due to the additional recitations in these claims. 
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In view of the above amendments and remarks, Applicants respectfully submit that 
claims 1-10 clearly define the present invention over the references relied on by the 
Examiner. Accordingly, reconsideration and withdrawal of the Examiner's rejections under 
35 U.S.C. § 103 are respectfully requested. 

CONCLUSION 

All the stated grounds of rejection have been properly traversed and/or rendered 
moot. Applicants therefore respectfully request that the Examiner reconsider all presently 
pending rejections and that they be withdrawn. 

It is believed that a full and complete response has been made to the Office Action, 
and that as such, the Examiner is respectfully requested to send the application to Issue. 

In the event there are any matters remaining in this application, the Examiner is 
invited to contact Paul C. Lewis, Registration No. 43,368 at (703) 205-8000 in the 
Washington, D.C. area. 
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If necessary, the Commissioner is hereby authorized in this, concurrent, and future 
replies, to charge payment or credit any overpayment to Deposit Account No. 02-2448 for any 
additional fees required under 37 C.F.R. §§ 1 .16 or 1 .1 7; particularly, extension of time fees. 

Respectfully submitted, 



BIRCH, STEWART, KOLASCH & BIRCH, LLP 




Paul C. Lewis, #43,368 
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[Name of the Document] Specification 

[Title of the Invention] Ball Motion Measuring Apparatus 
[What is claimed is] 
[Claim 1] 

A ball motion measuring apparatus comprising: 

a CCD camera for photographing a flying ball to obtain 
original image data; 

a calculating section for carrying out a magnifying 
process over a part including a ball image in an original image, 
thereby calculating magnified image data; and 

a display section for displaying a magnified image based 
on the magnified image data. 

[Claim 2] 

A ball motion measuring apparatus comprising: 
a CCD camera for photographing a flying ball to obtain 
original image data; and 

a calculating section for correcting a coordinate error 
made by a distortion of an original image which is caused by 
a lens of the CCD camera, thereby calculating correction data. 
[Claim 3] 

The ball motion measuring apparatus according to claim 
2, wherein the coordinate error is corrected based on a 
correction ratio determined by a distance from a center of the 
original image. 

[Claim 4] 

A ball motion measuring apparatus comprising: 
a CCD camera for photographing a flying ball to obtain 
original image data; and 

a calculating section for correcting a coordinate error 
made by a distortion of an original image which is caused by 
a lens of the CCD camera, thereby calculating correction data. 
[Claim 5] 

The ball motion measuring apparatus according to claim 
4, wherein the correction of the coordinate error serves to 
convert data obtained from an original image into data obtained 
by photographing a front part of the ball at infinity . 
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[Claim 6] 

The ball motion measuring apparatus in any one of claims 
1 to 5, wherein the CCD camera has a horizontal view angle of 
10 degrees or more. 

[Detailed description of the invention] 
[0001 1 

[Field of the invention] 

The present invention relates to a motion measuring 
apparatus for a ball such as a golf ball or a tennis ball. More 
particularly, the present invention relates to an apparatus for 
measuring the motion of a ball by utilizing image data obtained 
by a CCD camera* 

[0002] 

[Description of the related art] 

When a golf ball is hit with a golf club, it flies with 
a so-called backspin. The backspin is a rotation setting a 
horizontal direction orthogonal to a hitting direction (which 
will be hereinafter referred to as a x 'z direction") to be an 
axis. A lift acts on the golf ball through the backspin, thereby 
increasing a flight distance of the golf ball. In some cases, 
the golf ball flies with a so-called sidespin. The sidespin 
is a rotation setting a vertical direction (which will be 
hereinafter referred to as a *y direction") to be an axis. The 
golf ball turns left (a draw ball for a right-handed golf player) 
or turns right (a fade ball for the right-handed golf player) 
due to the sidespin . Furthermore, the golf ball sometimes flies 
with a rotation setting a horizontal direction identical to the 
hitting direction (which will be hereinafter referred to as an 
M x direction") to be an axis. 

[0003] 

A direction of the spin and a rotating speed greatly 
influence the subsequent trajectory of the golf ball. 
Similarly, the flight direction and flight speed of the golf 
ball greatly influence the subsequent trajectory. The 
measurement of the spin, the flight direction and the flight 
speed is effective for diagnosing the swing form of a golf player. 
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Moreover, the measurement is also effective for the evaluation 
of a golf ball and a golf club. In a stage in which the golf 
ball and the golf club are developed, the measurement is 
inevitable. 

[0004J 

Japanese Unexamined Patent Publication No. Hei 3-210282 
(1991/210282) has disclosed an apparatus for photographing a 
flying sphere twice by means of a camera and measuring the flight 
speed of the sphere based on two images thus obtained, in the 
apparatus, the moving distance of the sphere is obtained from 
the two images and is divided by a photographing time interval 
so that the flight speed is calculated. 
[0005] 

Japanese Patent No. 2810320 has disclosed a measuring 
method for photographing a flying golf ball twice at a 
predetermined time interval and calculating a spin rate from 
two images thus obtained. In the measuring method, a 
recognition mark put on the surface of the golf ball is read 
from first and second images and a rotating angle from the first 
image to the second image is obtained based thereon. The spin 
rate is calculated by the rotating angle and the photographing 
time interval. 

[0006] 

Japanese Unexamined Patent Publication No . Hei 10-186474 
(1998/186474) has disclosed a measuring method for 
photographing a flying sphere by means of two cameras in a 
shifted timing and calculating a spin rate, a flight speed and 
a flight direction from two images thus obtained. 
[0007] 

[Problem to be solved by the invention] 

In the case in which the flying golf ball is to be 
photographed by means of a camera, it is necessary to predict 
the position of the golf ball when a predetermined time passes 
after hitting and to determine the position of the camera such 
that the same position is included within a visual field. The 
head speed of a golf club is greatly varied depending on a golf 
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player. Moreover, there are various golf clubs from a driver 
(Wl) to a wedge. A head speed and a launch angle are greatly 
varied depending on the type of the golf club. Accordingly, 
the position of the golf ball after a predetermined time passes 
since hitting is considerably changed for each golf player or 
each used club. Therefore, it is not easy to predict the 
position of the golf ball after the predetermined time passes 
since the hitting. Even if the prediction can be carried out, 
a work for often changing the position of the camera is 
complicated. 

[0008) 

By using a camera having a wide-angle lens (that is, a 
camera having a great angle of view) f the visual field range 
is increased. Also in the case in which a hitting golf player 
take turns or a golf club to be used is changed, therefore, it 
is not necessary to often change the position of the camera. 
[0009] 

However, a ball in an image obtained by the camera having 
a great angle of view is reduced. Accordingly, in the case in 
which an operator specifies a predetermined point of the ball 
image through an input pen or the like to measure a spin rate, 
there is a problem in that precision in the pointing is 
deteriorated* Moreover, the image obtained by the camera 
having a great angle of view has a considerable distortion in 
a peripheral portion (a portion having a great distance, from 
the center of the image) * Therefore, true coordinates are 
considerably shifted from apparent coordinates on the image so 
that the precision in a measured value becomes insufficient in 
some cases* In many cases, furthermore, the ball is not 
positioned on the center of the image in the photographing using 
the camera having a great angle of view. For this reason, the 
front part of the ball cannot be photographed but the ball is 
photographed in an oblique direction. Thus, the true 
coordinates are considerably shifted from apparent coordinates 
on the image so that precision in the measured value becomes 
insufficient. 



6 



[0010] 

In consideration of such problems r it is an object of the 
present invention to provide an apparatus for measuring the 
motion of a ball with high precision. 
[00111 

(Means for solving problem] 

In order to achieve the object, the present invention 
provides a ball motion measuring apparatus comprising:' 

a CCD camera for photographing a flying ball to obtain 
original image data; 

a calculating section for carrying out a magnifying 
process over a part including a ball image in an original image, 
thereby calculating magnified image data; and 

a display section for displaying a magnified image based 
on the magnified image data. 

[0012] 

In the ball motion measuring apparatus, only a portion 
including the ball image in the original image is magnified and 
displayed on the display section. Accordingly, in the case in 
which an operator is to specify the predetermined point of the 
ball image, precision in pointing is high. Consequently, a 
measured value {such as a spin rate, a flight speed or a flight 
direction) can be obtained with high precision. This process 
will be hereinafter referred to as a ^magnifying process". 
[0013] 

In order to achieve the object, moreover, another 
invention provides a ball motion measuring apparatus 
comprising: 

a CCD camera for photographing a flying ball to obtain 
original image data; and 

a calculating section for correcting a coordinate error 
made by a distortion of an original image which is caused by 
a lens of the CCD camera, thereby calculating correction data. 
[0014] 

In the ball motion measuring apparatus, the shift of 
apparent coordinates on the original image from true 
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coordinates is corrected* More specifically, correction data 
are obtained by modifying the distortion in the peripheral 
portion of the original image which is caused by the lens. 
Accordingly, the ball motion can be measured with high precision. 
The correction will be hereinafter referred to as ^distortion 
correction". 

[0015] 

It is preferable that the coordinate error should be 
corrected based on a correction ratio determined by a distance 
from a center of the original image* Consequently, the 
precision in measurement can further be enhanced. 
[0016] 

In order to achieve the object, moreover, a further 
invention provides a ball motion measuring apparatus 
comprising: 

a CCD camera for photographing a flying ball to obtain 
original image data; and 

a calculating section for correcting a coordinate error 
made by a shift of a direction of a ball image from a direction 
of an optical axis of the CCD camera, thereby calculating 
correction data. 

[0017] 

Also in the ball motion measuring apparatus, the shift 
of the apparent coordinates on the original image from the true 
coordinates is corrected. Accordingly, the ball motion can be 
measured with high precision. More specifically, the data 
obtained from the original image can be converted into data 
obtained by photographing the front part of the ball at infinity. 
The correction will be hereinafter referred to as xx oblique 
correction". 

[0018] 

The process or correction is particularly effective for 
measurement to be carried out by means of a camera having a 
horizontal view angle of 10 degrees or more by using a wide-angle 
lens. As a matter of course, also in the case in which a standard 
lens or a telephoto lens is used, an enhancement in the precision 
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can be obtained. The "horizontal view angle of 10 degrees" 
implies that a photographic range in a horizontal direction is 
±5 degrees to a forward direction. 
[0019] 

[Description of the preferred embodiments] 

The present invention will be described below in detail 
based on a preferred embodiment with reference to the drawings. 
[0020] 

Fig. 1 is a plane view showing an example of a ball motion 
measuring apparatus of the present invention. Further, Fig. 2 
is a front view of the ball motion measuring apparatus. The 
present ball motion measuring apparatus serves to measure the 
motion (a spin rate, a flight speed, a launch angle and the like) 
of a golf ball. The ball motion measuring apparatus comprises 
a tee 3 for mounting a golf ball 1 thereon, a first camera 5, 
a second camera 7, a pair of first stroboscopes 9, a pair of 
second stroboscopes 11, a pair of optical sensors 13, a CPU 15 
to be a calculating section, and a monitor 17 to be a display 
section. The first camera 5 and the second camera 7 are CCD 
cameras having a shutter function. The CCD camera comprises 
a wide-angle lens and has a wide photographic range {a 
horizontal view angle of 10 degrees or more) , Usually, a camera 
having a horizontal view angle of 150 degrees or less is used. 
The shutters of the first camera 5 and the second camera 7 are 
opened or closed by a control section (not shown) receiving a 
trigger signal which will be described below. 
[0021] 

When the motion of the golf ball 1 is to be measured by 
the ball motion measuring apparatus, the golf ball 1 is first 
mounted on the tee 3 and a golf player 19 swings a golf club 
21. Consequently, a head 23 passes through a portion just above 
the optical sensor 13. A head speed can be calculated from a 
deviation of the times at which the head 23 passes through the 
portions just above the two optical sensors 13 and 13. By the 
passage of the head 23, the trigger signal is generated from 
the optical sensor 13. 
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[00221 

Then, the head 23 collides with the golf ball 1. The golf 
ball 1 is launched in an obliquely upper right direction in Fig. 
2. After a predetermined time passes since the optical sensor 
13 generates the trigger signal, the shutter of the first camera 
5 is opened . Moreover, the first stroboscope 9 emits light 
synchronously with the opening operation of the shutter. 
Consequently, an original image is photographed by means of the 
first camera 5 ♦ After a predetermined time further passes since 
the first camera 5 carries out the photographic operation, the 
shutter of the second camera 7 is opened- Moreover, the second 
stroboscope 11 emits light synchronously with the opening 
operation of the shutter. Consequently, an original image is 
also photographed by means of the second camera 7. Both of the 
original images are static images. Data on the respective 
original images are sent to the CPU 15. The CPU 15 carries out 
a process which will be described below in detail. 
[0023] 

The positions of the first camera 5 and the second camera 
7 are determined such that the flying golf ball 1 can be 
photographed. The positions of the cameras 5 and 7 are usually 
determined such that the golf ball 1 can be photographed in 
almost the central part of the original image when the golf 
player 19 having a average head speed swings the average golf 
club 21. Consequently, the angles of view of the cameras 5 and" 
7 are increased, and furthermore, the golf ball 1 can be 
photographed on almost all original images even if the golf 
player 19 takes turns or the golf club 21 is changed. The ball 
motion measuring apparatus may be constituted such that a sound 
sensor for detecting a hitting sound is provided to generate 
a trigger signal* 

[0024] 

Fig. 3(a) shows an original image photographed by means 
of the first camera 5. Since the angle of view of the first 
camera 5 is great, a ball image 25 is photographed to be small 
in the original image. Fig. 3(b) shows a magnified image 
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obtained by carrying out a magnifying process over the original 
image data. The magnifying process serves to magnify a 
rectangular portion (a portion including the ball image 25 in 
the original image) surrounded by a two-dotted chain line in 
the original image of Fig. 3(a). The monitor 17 displays the 
magnified image, 

[00251 

As shown in Fig. 3(b), a black recognition mark 27 is 
provided on the surface of the golf ball 1, The operator points 
the recognition mark 27 on the screen of the monitor 17. An 
input pen or the like is used for the pointing. By the pointing, 
a read value is read. Since the ball image 25 is magnified and 
displayed, a read value error is made during the pointing with 
difficulty. Similarly, the original image photographed by 
means of the second camera 7 is also subjected to the magnifying 
process so that a magnified image is displayed on the monitor 
17. Then, the recognition mark 27 is pointed. 

[0026] 

The rotating angle of the golf ball 1 from the 
photographing operation of the first camera 5 to that of the 
second camera 7 is obtained from the read values of two magnified 
images. The spin rate of the golf ball 1 is calculated from 
the rotating angle and a time interval from the photographing 
operation of the first camera 5 to that of the second camera 
7. Since the precision in the pointing is high, data on the 
spin rate thus obtained also have high precision. The launch 
angle, the flight speed and the like may be measured by the 
pointing. Also in this case, the data have high precision by 
the magnifying process. 

[0027] 

The original image photographed through the lens 
generates a distortion. In particular, in the case in which 
a camera having a great angle of view is used, the distortion 
is high. Fig. 4 shows a sample image 29 obtained by 
photographing a sample plate having a lattice pattern through 
the first camera 5. As is apparent from Fig. 4, a square 31 
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in the vicinity of the center of the sample image 29 has a contour 
shape close to a quadrate and the shape of a square 33 in a 
peripheral portion is changed. As a distance from the center 
of the sample image 2 9 is increased, the degree of the distortion 
becomes higher. Apparently, an error is made between apparent 
coordinates in the original image data and true coordinates when 
the golf ball 1 is photographed by means of the first camera 

[0028] 

The coordinates {true coordinates) in the sample plate 
are grasped in advance . A predetermined point is extracted from 
the sample image 29 (the image shown in Fig. 4) and is subjected 
to the pointing so that an error range between the apparent 
coordinates and the true coordinates can be obtained. Based 
on the error range, a correction ratio is calculated > The 
correction ratio is not uniform but is varied depending on the 
distance from the center of the image. In other words, the 
correction ratio is more increased in a portion closer to the 
periphery* 

[0029] 

Based on the correction ratio, the read value obtained 
from the original image data is subjected to a distortion 
correcting process. Consequently, correction data are 
obtained. Similarly, the read value obtained from the original 
image data of the second camera 7 is also subjected to the 
distortion correcting process. Consequently, correction data 
are obtained. The motion of the golf ball 1 can be measured 
from the two correction data with high precision. 
[0030] 

Fig. 5 is a plan view showing a state in which the 
photographing operation of the first camera 5 is carried out. 
In Fig. 5, a line f indicates a direction of an optical axis 
of the first camera 5 and a line s indicates a reference plane. 
The reference plane is extended in an x-y direction. The lines 
f and s are orthogonal to each other on a reference point 0. 
A point P indicates a focal point of a lens, a point Q indicates 
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a position of the recognition mark 27 provided on the surface 
of the golf ball 1, and a point R indicates a point obtained 
by projecting the point Q onto the reference plane. The golf 
ball 1 flies in almost the x direction (a right direction in 
Fig. 5) slightly upward. In the original image photographed 
in the state shown in Fig. 5, the direction of the ball image 
seen from the first camera 5 is shifted from the direction of 
the optical axis. The ball image is positioned on the right 
part of the original image. 

[00311 

A point TO is the center of the golf ball 1, When' the 
reference point 0 is an origin, a true x coordinate of the center 
TO is *a". In the original image data, one of ends of the golf 
ball 1 is regarded as a point Tl and the other point is regarded 
as a point T2, Accordingly, an intermediate point T3 of the 
points Tl and T2 is regarded as the apparent center of the golf 
ball 1, When the reference point 0 is an origin, an x coordinate 
of the apparent center of the golf ball 1 is regarded as *a'". 
More specifically, an error is made between the apparent 
coordinates in the original image data and the true coordinates 
(assuming that the front part of the golf ball 1 is photographed 
at infinity) - The true x coordinate of "a" can be calculated 
by the following equation: 

a = a' X {L 2 - r 2 ) / L 2 

wherein a distance of a segment PO is represented by L and a 
radius of the golf ball 1 is represented by r. 
[0032] 

Since the golf ball 1 is launched obliquely upward, a y 
coordinate of the center of the golf ball 1 is not zero. A true 
y coordinate of "b" can be calculated by the following equation: 

b - b' X (L 2 - r 2 ) / L 2 
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wherein an apparent y coordinate in the original image data 
photographed by means of the first camera 5 is represented by 
"b r " and an actual y coordinate is represented by "b". 
[0033) 

Thus, the read values (a r and b' ) obtained by the original 
image data are converted into the correction data (a and b) 
obtained by photographing the front part of the ball so that 
true coordinates (a, b, 0) of the center of the golf ball 1 are 
determined. 

[0034] 

A read value obtained from the original image data of the 
second camera 7 is also subjected to oblique correction. The 
flight speed,, the launch angle and the like of the golf ball 
1 are measured with high precision from the correction data of 
the first camera 5 and the correction data of the second camera 
7. In some cases in which the swing track of the golf player 
19 is not straight or a misshot is made, the center of the golf 
ball 1 is shifted from the reference plane. More specifically, 
a 2 coordinate of the center of the golf ball 1 is not zero in 
some cases. In these cases, it is preferable that the 
correction should be carried out in consideration of a shift 
in the z direction. A width of the shift in the z direction 
can be calculated from the size of the ball image in the original 
image, for example. 

[0035] 

In the case in which it is assumed that the point R has 
coordinates of {u' r v' , 0) and the center TO of the golf ball 
1 is positioned on the optical axis f , the position of the point 
Q is set to be a point Q 0 * Moreover, it is assumed that the 
coordinates of the point Q 0 are (x 0 , yo/ z 0 ) when the center of 
the golf ball 1 is an origin. Moreover, it is assumed that a 
length of a segment PQ is set to be *k" times as great as that 
of a segment PR . The point Qo is provided on a spherical surface . 
Therefore, a relationship expressed by the following equation 
(I) is established. 
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X 0 2 + y 0 2 + 2 0 2 - r 2 — (I) 

Moreover, since the coordinates of the point Q are (x 0 
+ a, y 0 + b f z 0 ) / the following equations (II) to (IV) are 
established. 

x 0 + a = k X u f (II) 

y 0 + b = k X v' — (III) 

zq - L = - (k X L) (IV) 

[0036] 

Since the values of x 'u' " and *v' " are read from the 
original image data, the value of u k" is calculated based on 
the equations (I) to (IV) . There are two solutions obtained 
from a quadratic equation for v 'k". One of the solutions 
corresponds to a point on this side of the reference plane s 

(closer to the CCD camera 5) and the other corresponds to an 
inner point of the reference plane s. A point at which the 
photographing is to be carried out is provided on this side of 
the reference plane s. Therefore, one of the two solutions 
which corresponds to the point on this side is employed to be 
the value of k. The values of x 0 , y 0 and z 0 are determined by 
the value of k and the equations (II) to (IV) . Similarly,, the 
values of x 0/ yo and z 0 are determined by the original image data 
obtained through the second camera 7, Based on the correction 
data, a spin rate can be obtained based on vector calculation. 
Japanese Patent No. 2810320 has disclosed a method for vector 
calculation as an example. Since the spin rate is obtained from 
the correction data after the oblique correction is carried out, 
precision thereof is high. 

[0037] 

While the ball motion measuring apparatus has such a 
structure that a ball image is displayed on the monitor 17 and 
an operator points a display screen to read a value, the value 
may be automatically read through an image processing or the 
like. In case of the automatic reading operation, the monitor 



15 



17 may be omitted . 
[0038] 

While two original images are photographed by means of 
two cameras in the ball motion measuring apparatus, three or 
more cameras may be provided* Moreover, the ball motion 
measuring apparatus may have such a structure that two or more 
ball images can be photographed on the original image 
photographed by means of one camera. 

[0039] 

While the magnifying correction, the distortion 
correction and the oblique correction can be carried out in the 
ball motion measuring apparatus, the ball motion measuring 
apparatus may have such a structure that any one or two of them 
can be performed. 

[0040] 

[Experiment 1] 

[Experiment Example A] 

A CCD camera having a wide-angle lens was prepared for 
first and second cameras. In the CCD camera, a visual field 
range in a horizontal direction is 250 mm (a horizontal view 
angle of 45.24 degrees) when a distance between a focal point 
P and a reference plane is 300 mm. The positions of both cameras 
are regulated such that a ball image was photographed in almost 
the central part of an original image and a golf ball mounted 
on a tee was hit. The flying golf ball was photographed in a 
shifted timing by means of two cameras so that two original 
images were obtained. The original images were displayed on 
a monitor and a flight speed, a launch angle, a backspin rate 
and a sidespin rate of the golf ball were calculated through 
an operation for pointing a recognition mark by an operator. 
The pointing operation and the calculation were repeated ten 
times to obtain a variation (4 x a ) „ 

[0041] 

[Experiment Example B] 

Original image data obtained in the experiment example 
A were subjected to a magnifying process having a double 
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magnification ratio so that a magnified image was obtained* 
Based on the magnified image, the same pointing operation and 
calculation as those in the experiment example A were repeated 
ten times. Thus, a variation (4 x a ) was obtained. 

[0042] 

[Table l] 

Table 1 Result of Experiment 1 





Experiment 
Example A 


Experiment 
Example B 


Type of lens 


Wide angle 


Wide angle 


Magnifying correction ratio 


No 


X 2 


Distortion correction 


No 


No 


Oblique correction 


No 


No 


Position of 
ball image in 
original image 


First camera 


Center 


Center 


Second camera 


Center 


Center 


Variation 


Flight speed 
(m/s) 


0.18 


0.11 


Launch angle 
(deg.) 


0.31 


0.15 


Backspin rate 
(rpm) 


180 


88 


Sidespin rate 
(rpm) 


168 


84 



[0043] 



In the Table 1, a measured value has a smaller variation 
in the experiment example B than that in the experiment example 
A, Consequently, it is apparent that precision in measurement 
can be enhanced by the magnifying process. 

[0044] 

[Experiment 2] 
[Experiment Example CJ 

The same first and second cameras as those in the 
experiment example A were prepared. The position of the first 
camera was regulated such that a ball image can be photographed 
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on the lower left part of an original image and the position 
of the second camera was regulated such that a ball image can 
be photographed on the upper right part of the original image. 
Then, a flying golf ball was photographed by means of the two 
cameras in a shifted timing. An original image thus obtained 
was subjected to a double magnifying process. Thus, two 
magnified images were obtained. The magnified images were 
displayed on a monitor so that a read value was obtained through 
an operation for pointing a recognition mark by an operator. 
A flight speed and a launch angle of the golf ball were calculated 
from the read value. • The pointing operation and the calculation 
were repeated ten times. Thus, a mean value was obtained. 
[0045] 

[Experiment Example D] 

The read value obtained by the pointing operation 
according to the experiment example C was subjected to oblique 
correction so that correction data were obtained. A mean value 
of each of a flight speed and a launch angle was obtained from 
the correction data. 

[0046] 

[Experiment Example E] 

The read value obtained by the pointing operation 
according to the experiment example C was subjected to 
distortion correction so that correction data were obtained. 
A mean value of each of a flight speed, and a launch angle was 
obtained from the correction data. 

[0047] 

[Experiment Example F] 

The read value obtained by the pointing operation 
according to the experiment example C was subjected to oblique 
correction and distortion correction so that correction data 
were obtained. A mean value of each of a flight speed and a 
launch angle was obtained from the correction data. 

[0048] 

[Reference Example] 

A first sensor through which a laser beam passes 
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vertically was provided in almost the same position as the 
position of the ball image of a first camera . Moreover, a second 
sensor through which a laser beam passes vertically was provided 
in almost the same position as the position of the ball image 
of a second camera. A time required for the passage of a golf 
ball from the first sensor to the second sensor was measured 
to obtain a flight speed. The flight speed approximates to a 
true value. Moreover, a line sensor through which a laser beam 
passes horizontally was provided ahead of the ball image of the 
second camera in the direction of flight. A launch angle was 
calculated based on a height of the golf ball passing through 
the line sensor and a height of the golf ball mounted on a tee. 
The launch angle approximates to a true value. 
[0049] 
[Table 2] 





Experiment 
Example C 


Experiment 
Example D 


Experiment 
Example £ 


Experiment 
Example F 


Ref. Ex. 
(true value) 


Type of lens 


Wide angle 


Wide angle 


Wide angle 


Wide angle 


Measurement 


Magnifying 
correction ratio 


x 2 


x 2 


x 2 


x 2 


By 

Laser 
sensor 


Distortion 
correction 


No 


NO 


Yes 


Yes 


Oblique 
correction 




No 


Yes 


NO 


Yes 


Position of 
ball image 
in original 
image 


First 
camera 


Lower left 


Lower left 


Lower left 


Lower left 


Second 
camera 


Upper 
right 


Upper 
right 


Upper 
right 


Upper 
right 




Flight speed (m/s) 


56. 5 


56.3 


58.0 


57.8 


57.8 


Launch Angle (deg. ) 


11.2 


11.2 


12.1 


12.1 


12.1 



[0050] 
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From the Table 2, it is apparent that the measured value 
approximates to a true value by the distortion correction. 
[00511 

[Experiment 3] 
[Experiment Example G] 

An XY table was prepared and provided on a reference plane 
s. Two golf balls were fixed to the XY table. The positional 
relationship between the two golf balls (the relationship 
including a backspin) was set to be almost equivalent tb that 
between two ball images in the case in which photographing was 
carried out twice in a shifted timing from the trajectory of 
the hit golf ball. On the other hand, the same first and second 
cameras as those in the experiment example A were prepared. The 
position of the first camera was regulated such that one of the 
golf balls can be photographed on the center of an original image 
and the position of the second camera was regulated such that 
the other golf ball can be photographed on the left part of the 
original image. Then, two original images obtained by the 
respective cameras were subjected to a magnifying process 
(double). Thus, two magnified images were obtained. The 
magnified images were displayed on a monitor so that a read value 
was obtained through an operation for pointing a recognition 
mark by an operator. The read value was subjected to distortion 
correction so that correction data were obtained. A backspin 
rate and a sidespin rate were calculated from the correction 
data . The pointing operation and the calculation were repeated 
ten times. Thus, a mean value was obtained. 
[0052] 

[Experiment Example H] 

A mean value of each of a backspin rate and a sidespin 
rate was calculated in the same manner as in the experiment 
example G except that distortion correction and oblique 
correction were carried out to obtain correction data. 

[0053) 

[Experiment Example I] 

A mean value of each of a backspin rate and a sidespin 
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rate was calculated in the same manner as in the experiment 
example G except that the other golf ball was moved without a 
rotation over an XY table and was photographed on the right part 
of an original image. 

[00541 

(Experiment Example J] 

A mean value of each of a backspin rate and a sidespin 
rate was calculated in the same manner as in the experiment 
example I except that distortion correction and oblique 
correction were carried out to obtain correction data. 

[0055) 

[Experiment Example K] 

A CCD camera having a telephoto lens was prepared for first 
and second cameras . The telephoto lens has a visual field range 
of 60 mm with a distance between a focal point P and a reference 
plane of 2300 mm. A golf ball put on an XY table was moved 
without a rotation and a position was regulated such that a ball 
image can be photographed on almost the center of an original 
image. Two original images obtained by the respective cameras 
were not subjected to correction but displayed on a monitor. 
The ball images thus displayed were sufficiently large. A 
backspin rate and a sidespin rate were calculated through an 
operation for pointing a recognition mark by an operator. The 
pointing operation and the calculation were repeated ten times 
so that a mean value was obtained. The backspin rate and 
sidespin rate thus obtained approximate to true values. 

[00563 

[Table 3] 
Table 3 Result of Experiment 3 
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Type of lens 



Magnifying 
correction ratio 



Distortion 
correction 



Oblique 
correction 



Position of 
ball image 
in original 
image 



First 
camera 



Second 
camera 



Backspin rate(rpm) 



Sidespin rate(rpm) 



Experiment 
Example G 



Wide angle 



X 2 



Yes 



No 



Center 



Left 



2183 



781 



Experiment 
Example H 



Wide angle 



x 2 



Yes 



Yes 



Center 



Left 



2410 



103 



Experiment 
Example I 



wide angle 



x 2 



Yes 



Mo 



Center 



Right 



3243 



-2323 



Experiment 



Wide angle 



Experiment 
Example K 



Telephoto 



x 2 



Yes 



Yes 



Center 



Right 



2399 



126 



No 



No 



No 



Center 



Center 



2420 



140 



[0057] 

From the Table 3, it is apparent that the measured value 
approximates to a true value through oblique correction also 
in the case in which the wide-angle lens is used. 
[0058] 

While the ball motion measuring apparatus according to 
the present invention has been described by taking the golf ball 
• as an example, the present invention can be used for measuring 
all balls such as a tennis ball. 
[0059] 

[Effect of the invention] 

As described above, an apparatus for measuring the motion 
of a ball may present a measurement with high precision. 
[Brief description of the drawings] 
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[FIG 1] 

Fig. 1 is a plan view showing a ball motion measuring 
apparatus according to an embodiment of the present invention, 

IPIG 2) 

Fig. 2 is a front view showing the ball motion measuring 
apparatus in Fig. l, 

(FIG 3] 

Fig. 3(a) is a front view showing an original image, 
Fig. 3(b) is a front view showing a corrected image, 
[FIG 4] 

Fig. 4 is a view showing a sample image in which a sample 
plate having a lattice pattern is photographed, and 
[FIG 5] 

Fig. 5 is a plan view showing the state of photographing 
carried out by means of a first camera. 

[Explanation of reference numerals] 

I. ..golf ball 
5...first camera 
7...second camera 
9...first stroboscope 

II. ..second stroboscope 
13...optical sensor 
15...CPU 

17..jnonitor 
25..iall image 
27...recognition mark 
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Fig. 1 
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9 5 




X 

Fig.2 
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[PIG 3] 




Fig. 3 
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Fig. 4 
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[FIG 5) 





(Name of the document] Abstract 
[Abstract] 

[Problem to be solved] 

The present invention is aimed to provide an apparatus 
for measuring the motion of a ball with high precision. 

[Method for solving the problem] 

A ball motion measuring apparatus comprises a tee 3 for 
mounting a golf ball 1 thereon, a first camera 5, a second camera 
7, a pair of first stroboscopes 9, a pair of second stroboscopes 
11, a pair of optical sensors 13, a CPU 15. to be a calculating 
section, and a monitor 17 to be a display section. The first 
camera 5 and the second camera 7 are CCD cameras having a shutter 
function. Photographing is carried out by means of the first 
camera 5 after a predetermined time passes since the golf ball 
1 is hit, and the photographing is carried out by means of the 
second camera 7 after a predetermined time further passes. A 
magnified image is formed by original image data thus obtained 
and a read value obtained by pointing the magnified image is 
subjected to distortion correction or oblique correction so 
that correction data are obtained. Based on the correction data, 
a flight speed, a spin rate or a launch angle of the golf ball 
1 is calculated. 

[Selected Drawing] FIG 1 
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Motion - It's something we experience all day every day, we see it, sometimes we hear it 
and sometimes we hear it and sometimes we, in various ways, feel the effects of it We use 
video recordings to store or remotely display such visual experiences. Yet, isn't it strange that, 
until now, no one treated video as a collection of captured motions? It sounds simple, in theory, 
but in practice, identifying the motion within a video is very difficult to do. 

A Video is composed of a sequence of consecutive pictures, called Frames, photographed 
in motion by a specially designed camera and shown on display in such a rapid succession as to 
give the illusion of natural movement of objects, background, and camera perspective. 

When a Digital Video is created, the individual Frames of the video are digitized by 
creating a grid of color dots that represent the frame. The individual dots of color are called 
Pixels. 

A unique way of looking at digital video has been developed. Simply stated, digital 
video is seen as a collection of motions. There are really two types of motion within a video: 

1 . Object Motion - such as a person in the foreground or a vehicle in the 
background. 
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2. Camera Motion - Panning and/or Zooming to change the perspective of 
the electronic eye capturing the video. 

A new technology employed in this invention is applied at the individual pixel level to 
identify the motion within a video. The technology can differentiate between themotion of each 
individual object within the video as well as motion resulting from movement of the camera. 
The ability to first separate the types of motion and then manipulate the size or position of an 
object, its direction, velocity and acceleration or the impact of the cameras movement enables an 
amazing set of applications and capabilities. 

Application to Security 

Clearly, the single most import feature associated with video surveillance is detecting 
motion. Often, a security staff is confronted with multiple, simultaneous video streams. Each 
has its own set of motions; each must be watched at all times. Current offerings may provide a 
warning when motion occurs, but this is really basic analysis of the video. With the new 
technology, a vast array of analysis can be performed on the incoming video and the actual 
display of the video can be modified to highlight important motion and eliminate operator 
overload. 

The following pages will illustrate unique motion technologies using examples common 
to video surveillance. The motion-based enhancements are grouped into three categories: 
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1 . Converting Panning Cameras to Stable Panoramic Display: The hypnotic movement of a 
panning camera can be transformed into a stable panoramic view of the entire scene. This 
reduces operator fatigue and keeps everything on-screen - even when the camera has moved 
to another position. 

2. Making Use of Object Motion: With motion identified, moving objects can be highlighted, 
enlarged, tracked and alarmed. More advanced techniques can even tell normal motion from 
unusual motion. 

3. Compressing Video for Remote Surveillance and archiving: Motion techniques can 
greatly increase the compression of video to allow transmission over remote modems as well 
as shrinking the storage required for archiving video. 

Image Stabilization 

A non-hardened camera platform can result in small random movements in the video that 
often make details difficult to distinguish. An Anti-Shake™ technology determines the random 
camera movement and eliminates it, resulting in smooth, easy to view motion. 

Pan & Zoom Governor 

In a stressful moment, an operator may move a camera too quickly to accurately view the 
video. With a technology employed in the invention, frames can be inserted between the 
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existing frames of video to enable viewing of these events. The pan and zoom governor is 
described in a copending application entitled A Method and Apparatus for Automatically 
Adjusting Video Panning and Zoom Rates, filed September 27, 2000 invented by Steven Edelson 
and Ralph CarballaL This application is hereby incorporated by reference. 

Panoramic Display of Sweeping Camera 

One type of surveillance camera installation involves having the camera sweep back and 
forth through a viewing field. This opens holes in security, as problems become "invisible" 
when the camera is facing away. The present invention can be applied to enable a constant 
panoramic view of the camera's sweep field to eliminate this problem. In fact, the monitor will 
always show the entire sweep of the camera as a static display that is constantly updated. Fig. 1 
illustrates how such a display screen may appear. 

Objects within a video are composed of pixels. The technology for analyzing video and 
determining a vector defining direction and velocity, for each pixel in a frame of video is 
disclosed in copending application serial no. 09/593,521 entitled System for the Estimation of 
Optical Flow, filed June 14, 2000, invented by Siegfried Wonneberger, Max Griessl, and Markus 
Wittkop. This application is hereby incorporated by reference. 

The set of vectors representing the pixel motion from frame to frame is called a dense 
motion vector field. By analyzing these vectors, we can identify and manipulate individual 
moving objects within the video. 
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Currently, cameras pan back and forth slowly so the operator can see a relatively stable 
scene. By using the panoramic view, the scene is always stable for the operator and the camera 
speed can be increased to give more rapid updates. Also, because the operator can see the entire 
viewing area of the camera's pan, there may be a time when he/she wants to get a- clear update on 
a specific object Rather than wait for the panning camera, a new implementation could allow 
the camera to snap to that view for real time display. 

When the camera is being panned, all of the background pixels will move at substantially 
the same velocity. As a result, dense motion vector fields can be used to determine the motion 
component due to camera motion simply by processing the dense motion field vector to 
determine the most commonly occurring vector, which will correspond to the camera motion 
component. This camera motion vector, which represents the motion of the camera that is 
distinct from the motion of objects, is used to determine location of each successive video frame 
in a panoramic display, each frame being displaced from the previous one by the camera motion 
vector. Assuming that the camera motion is horizontal panning, the successive frame will be 
located in an overlapping relationship distributed horizontally across the screen. Each new frame 
will overwrite the overlapped portion of the preceding frame on the panoramic display so that a 
vertical strip of the preceding frame remain displayed adjacent to edge of each new frame. 

Making Use of Object Motion 
Enhancing the Display of Moving Objects 



5 



36719:167380 
DC2DOCS 1:247726 



Motion detection, as an enhancement to video surveillance systems is nothing new. 
However, the motion-based approach of this invention not only enhances this capability, but also 
enables the user to change the way that moving objects are displayed on the video. 

To analyze actual object motion not caused by camera motion, the camera? motion vector 
is subtracted from each vector of the dense motion vector field. This calculation results in a 
dense motion vector field representing only actual motion of objects, that is object motion net of 
camera motion. 

To identify a moving object, the dense motion vector field net of camera motion is 
analyzed to find a set of adjacent motion vectors which are the same or similar. The pixels for 
such vectors will be the pixels of a moving object. An object motion vector representing the 
motion of the object can then be determined as a vector corresponding to the most commonly 
occurring vector in the set 

Using object identification and motion estimation techniques, objects in the viewing field 
can be moved to approximate their location based on their last viewed object motion vector. 
When an object is again in the view of the camera, its position is updated. 

Depending on settable movement limits, a variety of enhancements can be applied to 
highlight the movement to the operator. 
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Highlight - Increase the brightness and contrast allowing moving objects to stand out 
against the background. 



> Color - Any color can be mixed to color the moving objects. Also, a colored object can 
have its saturation boosted or lowered to contrast with the background. -* 

» Haloing - A bright halo in white or any color can be projected around the moving object. 

• Binocular Effect - An area surrounding the object can be defined to be magnified. This 
area will have a zoom effect, magnifying the area by 2x or 4x while leaving the area 
outside the area unchanged. 

• Background Effects: In addition to enhancing objects in the display, the background can 
be modified to further differentiate the moving objects. This can include converting a 
color background to black & white or applying a defocus filter to the background. 

The technology in detecting depicted moving objects and changing the display 
characteristics of the moving objects is disclosed in a copending application entitled Special 
Effects in Video in Response to Depicted Motion, filed September 11, 2000, invented by Steven 
D. Edelson. This application is hereby incorporated by reference. In addition to highlighting a 
moving object, it is particularly useful in surveillance applications to highlight an object, which 
was moving, but which is no longer moving. 
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Suspect & Expected Motion : 

Current motion detection systems have no ability to accurately differentiate between 
suspect and expected motion. They use defined zones that are ignored to deal with areas that 
will experience regular, expected motion. This inherently leaves a security hole where suspect 
motion can occur. 

With the present invention, the operator can identify the motion of objects that is 
expected. These objects might be displayed with a blue hue. Should an unexpected motion 
occur in this area, it would still be flagged, for instance by haloing the object in red, for operator 
attention. Thus, the area is still under enhanced surveillance. Additionally, using object size and 
motion information, a system might be produced to enhance rejection of trivial movement such 
as that of a small animal 

To distinguish expected motion from unexpected or suspect motion, the dense motion 
vector field representing the expected motion is inputted and stored. This can be done by taking 
a video of the expected motion, which might be of a rotating fan or rotating rotisserie and storing 
the dense motion vector field representing the expected motion. The surveillance video dense 
motion vector fields are then compared with the motion vector field representing the expected 
motion. Any motion vectors, which do not correspond to the motion vectors of the expected 
motion, represents unexpected motion or suspect motion and the moving objects corresponding 
to the unexpected motion is highlighted as suspect motion. 
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The degree of dissimilarity with the expected motion required for the moving object to be 
tagged as suspect motion can be varied depending upon the type of expected motion. If the 
expected motion in constant and unvarying as in a rotating fan, then any substantial variation 
from the expected motion is tagged and depicted as suspect. If the expected motion is more 
varied as people walking along a path or a punch press operation, the fields of vectors 
representing the expected motion and representing the motion question are analyzed and 
compared to see if two motions are generally the same. If not the motion is tagged as 
unexpected motion. 

High Resolution Visual Bubble: 

A motion-isolating technology provides an ability to identify a key area of a video that 
benefits from high-resolution viewing. (See Figure 2). On detecting motion, the system can be 
set to automatically define a high resolution "Viewing Bubble" around the suspect motions. If 
desired, the operator can be given the ability to electronically steer this bubble around the scene 
to more clearly view items of interest. 

Crop & Zoom 

Another technology allows cropping of a video in an area of interest generally followed 
by a zoom to expand the area to full screen. The zoom can also be applied to any section of the 
video. 
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Slow Motion Effect 

Applying frame interpolation to a full 30fps video to create more frames between existing 
frames results in a synthetic slow motion effect. This is great for catching motion that is difficult 
to see at full speed. 

The technique of providing slow motion is disclosed in copending application Serial No. 
9/459, 987 filed December 14, 1999 entitled Slow Motion System. This application is hereby 
incorporated by reference. 

Anticipating Object Motion 

An object in a digital video is a collection of pixels, each of which is assigned a vector 
defining direction and velocity. By analyzing the vectors of an object's pixels, a motion vector 
for the object can be determined as described above. With this vector, software can make 
estimation of the probable location of the object in the near future. 

Display of Anticipated Object Motion: 

The invention provides the ability to anticipate the movement of an identified object 
within the video. This can be utilized in at least three ways. First, the display can have an arrow 
added to the object to indicate the direction and velocity as shown in the Fig. 3. In this example, 
the subject pushing a cart is moving at 2.6 feet per second at a heading of 045°. Second, the 
algorithms can estimate a probable location for the object at some time in the future based on its 
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current motion. This location could be communicated with a graphic object on the display as 
well. Finally, a number of statistics for the object can be gathered such as time in the display, 
time from operator flagging until the object leaves display, etc. 

Automatically Panning a Camera to Follow Movement: 

Once an object has been identified and flagged, it is possible to make use of the object 
motion vector to move the camera and keep the object centered. This technology could include 
the ability to anticipate the probable future location of the object and move the camera to that 
location. 

Video Data Reduction And Frame Interpolation 

In a copending application Serial No. 09/459, 988 filed December 14, 1999 entitled 
Motion Picture Enhancing System, a technology is disclosed for generating frames to be 
interpolated between adjacent frames of a video which is at a slow frame rate to eliminate the 
jerkiness in the video display. This application is hereby incorporated by reference. The 
interpolating process of this application is called MediaMend™. In a copending application filed 
July 1 7, 2000 entitled Method and Apparatus for Reducing Video Data, invented by Steven D. 
Edelson, there is disclosed a system for scoring the video frames to determine which frames can 
be recreated most accurately by the MediaMend™ process, and then eliminating those frames to 
reduce the video data. This application is hereby incorporated by reference. 

Compression for Transmission of a Remote Surveillance Camera to Base Facility 
The Problem: 
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Many companies are using surveillance cameras to observe remote locations as an 
alternative to expense of basing a human at the location or sending someone out when there is an 
apparent problem. The surveillance video is commonly transmitted over a telephone line via 
modem or over the Internet 

Both the telephone and the Internet share a common problem when transmitting video-a 
distinct lack of bandwidth. The limited available bandwidth does not allow full-size, full-speed 
video to be transmitted. As a result, current products use compression routines to shrink the size 
of video prior to transmission. 

Unfortunately, most video compression routines for phone or the Internet begin by 
shrinking the video by simply removing up to 80% of the existing frames. As a result video sent 
over the Internet or by phone is very jerky. This jerkiness makes interpreting the content of an 
important surveillance video difficult and error-prone. 

The Solution: 

A security video transmission system based on the invention can significantly reduce 
bandwidth utilization and differentiate itself with two significant improvements over current 
solutions: 
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1 . When the video's background is static, the technology permits sending only the objects 
that are moving, not the background. The dense motion vector fields net of camera motion are 
used to identify the moving objects. The stationary background is transmitted only infrequently 
relative to the moving objects. Bandwidth is saved because the moving objects occupy much 
less of the scene than the stationary background and therefore the moving objects^are represented 
by much less data. Small objects separated from the background for transmission are called 
sprites. In accordance with the invention, the sprites may be automatically magnified before or 
when they are superimposed on the background. 

2. Rather than randomly eliminated frames on the camera side, the technology can analyze 
the video and discard the frames that can be recreated most accurately by MediaMend™ during 
playback on the receiving computer as described in the above mentioned copending application, 
Apparatus For Reducing Video Data, filed July 17, 2000. These techniques actually create a 
video that is smoother (full 30fps), clearer, and smaller than existing systems. 

Time Lapsed Storage and Archiving: 

The technology of the invention enables a higher level of storage efficiency. The storage 
of only moving areas can exponentially shrink storage needs. The dense motion vector fields 
identify the moving areas and then only these identified moving areas are stored. The ability to 
re-insert frames removed for compression, using MediaMend™ can greatly improve the 
playback viewing. 

DC2/247726 
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Stable View from a Scanning Camera: Note the radar-like 
highlighted area. This indicates the current view of the camera. As 
the camera moves, the view area is constantly updated. For a larger 
image, the scene can be split between two or more monitors. 



